REMARKS 

Applicants have now had an opportunity to carefully consider the Examiner's 
comments set forth in the Office Action of July 8, 2004. 

All of the Examiner's objections and rejections are traversed. 
Reexamination and reconsideration are respectfully requested. 

The Office Action 

The objection to the specification was withdrawn in view of the amended 
specification. 

Claims 1-3, 7, 8, 11 and 12 stand rejected under 35 U.S.C. §102(a) as being 
disclosed by U.S. Patent Application Document No. 20030007018 for Seni et al. 
(hereinafter Seni). 

Claim 4 stands rejected under 35 U.S.C. §1 03(a) as being unpatentable over 
Seni et al. as applied to claim 1 , and further in view of Official Notice. 

Claims 5 and 9 stand rejected under 35 U.S.C. §1 03(a) as being unpatentable 
over Seni et al. as applied to claim 1 , and further in view of U.S. Patent No. 5,889,523 
issued to Wilcox et al. (hereinafter Wilcox). 

Claims 14, 15, 17-20, 22 and 23 stand rejected under 35 U.S.C. §103(a) as 
being unpatentable over Seni et al. as applied to claim 1, and further in view of U.S. 
Patent No. 6,389,435 issued to Golovchinsky et al. (hereinafter Golovchinsky). 

Claim 21 stands rejected under 35 U.S.C. §1 03(a) as being unpatentable over 
Seni et al. as applied to claim 1 , further in view of Golovchinsky, and further in view of 
Official Notice. 

Claim 16 stands rejected under 35 U.S.C. §1 03(a) as being unpatentable over 
Seni et al. as applied to claim 1 , further in view of Golovchinsky, and further in view of 
U.S. Patent No. 6,470,095 issued to Mahoney et al. 

Claims 6, 10 and 13 stand objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

Claims 1-23 remain in this application. 

The Art Rejections 

In the Examiner's Response to Arguments with reference to claim 1, the 
Examiner states that claim 1 does not distinguish between "structured object 



8 



representations of the digital ink image" and Seni's [graphical handwriting user interface 
with text recognition], even when focus is directed to the instant specification where 
digital ink images are defined as symbols (i.e. structured object representations) and 
not converted into online images. However, while Seni discloses a user interface 
technique for handwriting entry on PDAs, Seni requires entering handwriting in a special 
area on the screen and, moreover, Seni deals with handwriting recognition for 
converting handwritten input into correlating text. Seni does not teach or suggest 
receiving a digital image having perceptually salient structures which can be 
automatically defined. At best, Seni requires the manual entry of one perceptually 
salient structure (word) at a time in the handwriting area 104 (paragraph 16). 

Further, Seni does not disclose or suggest any capability with respect to line art, 
only using the handwriting area for input of handwritten text, and attempting to 
recognize the text with the recognized text being displayed at the top of the interface 
display 102 (paragraph 16). 

Embodiments of the present application, as recited in claim 1, as amended, 
convert a digital image to structured object representations, each correlating to an 
automatically defined perceptually salient structure of the digital ink image. Further, 
each perceptually salient structure includes at least one of text and line art (see also 
paragraphs 35-41). On the other hand, as known in the art, recognized handwriting is 
presented as formal typed text, and is no longer a representation of the image in its 
original form, but rather a representation of characters that an application program 
recognizes as being in the image. Recognized line art is, according to embodiments of 
the present application, converted to a representation of a distinct area of the image 
which may be viewed by the user as an image, but is editable by a text/graphics editor 
(paragraphs 35-36 and 40). 

The concept of editing structured object representations converted from a 
received digital ink image, as recited in claim 1 and, as further described in paragraph 
40 of the present application, is not taught by Seni in paragraphs 22-23. While there is 
a reference to editing operations in these paragraphs, the editing is limited to selection 
of corrected replacement letters or words. There is no teaching that these editing 
operations are on the structured object representation of an automatically defined 
perceptually salient structure as taught in the present application and recited in the. 
limitations of claim 1, as amended. 

With reference to dependent claim 3, the Examiner likened "HAPPY" under 
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humeral 1 1 6 of FIG. 1 of Seni to a formal structured object representation as recited in 
the claim, and the Examiner further likened the handwritten script depicted at numeral 
104 to an informal structured object representation as recited in the claim. As clearly set 
forth in paragraph 43 of the present application, "formal" graphical material refers to 
geometrically precise or exact renditions of linework and shapes, while "informal" 
graphical material refers to approximate or geometrically imperfect figures that human 
observers nonetheless normally identify with their precise counterparts. These 
definitions allow for some imprecision that does not limit the functionality of the 
concepts described in the specification. For example, as described in paragraph 45, it 
is possible that a user may have any one of several intentions in importing hand-drawn 
material into a structured graphics editor and may make changes to the material 
accordingly. An example shown in FIG. 3B illustrates maintaining handwritten material 
as is, but changing the dashing and line thickness of a hand-drawn rectangle. FIG. 3C 
demonstrates a user rearranging rows of handwritten text. FIGS. 3D-3G provide even 
more examples showing how a user may selectively alter both informal and formal 
structured objects. On the other hand, there is no teaching or suggestion in Seni 
indicating that the informal representation of "happy" shown with numeral 1 04 of FIG. 1 
can be edited by the user. For example, there is no suggestion that the user could 
select either handwritten "p" and replace it with another letter in the informal 
representation. Seni only discloses changes to the formal form of happy, i.e. the 
converted text (which would be in a converted digital format such as ASCII) whereas 
claim 3 of the present application clearly recites that both the formal and the informal 
structured object representations are editable. 

With reference now to dependent claim 8, the Examiner states that "Said sub- 
group of all the structured object representations corresponds to a letter or character of 
a word or multi-digit number." Applicants respectfully traverse the Examiner's statement. 
As recited in the claim, "the editing by the structured text/graphics editor permits 
movement of structured object representations by at least one of, individual objects, a 
sub-group of all the structured object representations, or as an overall group of the 
structured object representations." Applicants submit that the letter or character of a 
word or multi-digit number that the Examiner refers to does not constitute a sub-group, 
but rather an individual object. Paragraph 42 of the present application describes how 
embodiments pick out primitive text and graphic elements and bitmap images, then 
assemble perceptually salient groupings of those elements. Paragraph 49, in 
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'describing the Alternative Graph, provides clarification of the differences between 
objects and groups when defining object nodes and group nodes. Groups are described 
as objects that are bound together as a group and treated as a single object. The single 
letter or character that the Examiner refers to does not comprise a collection of objects, 
but, rather, comprises only a single object, unlike the groupings and sub-groupings of 
the present application as recited in dependent claim 8. 

In rejecting dependent claim 4, the Examiner states that the art is replete with 
importing capabilities to configure structured object representations to represent an 
electronic slide. Dependent claim 4 recites an additional limitation to base claim 1 
wherein "the step of converting the digital ink image into structured object 
representations of the digital ink image includes configuring the structured object 
representations to represent an electronic slide of the structured text/graphics editor." 
The structured object representations, however, according to claim 1, as amended, 
correlate to an automatically defined perceptually salient structure of the digital ink 
image, each perceptually salient structure including at least one of text or line art. 
Applicants are not aware of any art available at the time of filing of the present 
application that imports automatically, perceptually structured informal digital ink into an 
electronic slide program. 

Applicants respectfully submit, for the above reasons, that independent claim 1 
patentably defines overSeni and, thus, is in condition for allowance, as are claims 2-10, 
depending therefrom. 

The Examiner interpreted independent claim 1 1 as a system for performing the 
method of claim 1 and, consequently, rejected claim 1 1 on the same grounds as claim 
1 1 . Applicants submit, therefore, that independent claim 1 1 , as amended, like claim 1 , 
patentably defines over Seni and is condition for allowance. Likewise, claims 12-13, 
depending from claim 11, are also in condition for allowance. 

With attention now to independent claim 14, the Examiner again uses Seni as 
applied to claim 1 in rejecting claim 14. Although acknowledging that Seni does not 
disclose "representations correlating to perceptually salient areas of the digital ink 
image," the Examiner applies Golovchinsky, col. 4, lines 58-67, to this claimed 
limitation. This section of Golovchinsky discusses providing a perceptually motivated 
model of freeform digital ink mark that applies higher weight to more saliently marked 
terms, and therefore uses somewhat similar language, however, this disclosure is 
teaching a substantially different concept than the present application. 
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Apparently, Golovchinsky applies a higher weight to terms that are marked with 
the bright color digital ink than those terms that are marked with a less salient digital ink. 
On the other hand, claim 14, as amended, recites an image having structured object 
representations correlating to automatically defined perceptually salient areas of a 
digital ink image. Whereas the structured object representations of claim 14 are noted 
to correlate to automatically defined perceptually salient areas of the digital ink image, 
Golovchinsky contains no concept of representing structure, or defining perceptually 
salient areas in existing digital ink. Rather, in Golovchinsky, perceptual salience lies in 
the colors of digital ink strokes, on an independent stroke-by-stroke basis. In the 
present application, perceptual structure is represented among collections of strokes. 
Further, in Golovchinsky, it is the user who must manually mark the salient areas of the 
document with the digital ink marks (col. 2, lines 2-5) while, on the other hand, in the 
present application (paragraph 40), and as recited in amended claim 14, the 
perceptually salient areas are automatically defined. 

To summarize, Golovchinsky refers to saliently marked terms, where the user 
explicitly gives weighting to written terms by deliberate selection of digital ink with 
salient characteristics, such as, e.g., a bright color. The present application, on the 
other hand, as recited in claim 14, as amended, recites structured object 
representations correlating to automatically defined perceptually salient areas of the 
digital ink image. Golovchinsky's manually marked terms are not comparable to the 
automatically defined perceptually salient areas recited in the claim. 

In rejecting dependent claim 15, the Examiner cites the ability described inSeni 
for a user to, at any time, select one (or more) characters) from a previously entered 
word and write a new characters) in the input area with the result replacing the 
selected text. According to the limitation recited in claim 15, however, it is the informal 
structured object representations which are editable to formal structured object 
representations. As described above with reference to claim 3, Seni only discloses 
changes to the formal converted text whereas claim 15, unlike Seni, recites editing of 
the informal structured object representations. 

With reference now to claim 1 7, the Examiner asserts that a letter within a word, 
as described in Seni and Golovchinsky, is a first structured object representation 
spatially contained within a second structured object representation. It is to be 
appreciated that the structured object representations recited in claim 1 7 are described 
in paragraph 37 as perceptually salient areas that ordinary viewers of graphic imagery, 
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text, or mixtures of these, would readily identify in certain groupings and collections of 
image primitives (i.e., pixels) as being sensible, visibly apparent or meaningful units on 
which to perform image editing operations. These groupings may be based on any of a 
variety of properties, including but not limited to spatial proximity, curvilinear alignment, 
connectedness, colinearity, forming a closed path, being contained within a closed or 
mostly-closed figure, forming elementary geometry shapes or configurations, and 
combinations of these. This description in the present application is obviously referring 
to groupings in a two-dimensional, spatial context. A letter within a word as described 
by Seni is only a positional grouping in a one-dimensional sense, unlike the two- 
dimensional, spatially contained sense recited in the subject claim. 

In rejecting dependent claim 19, the Examiner asserts that the structured object 
representations defining a text block structure are disclosed by Seni and Golovchinsky. 
Seni, however, does not disclose finding textual block structure in informal digital ink 
images but, rather, only deals with one word at a time as shown in FIG. 1 . On the other 
hand, claim 19 recites that "the structured object representations define a text block 
structure," wherein, from base claim 14, as amended, each structured object 
representation correlates to an automatically defined perceptually salient area of the 
digital ink image. As further described in paragraphs 44-45 and 56 of the present 
application, the subject claim is dealing with rows and columns comprising multiple 
words in automatically defined perceptually salient areas of the digital ink image. Seni 
and Golovchinsky do not teach or suggest such features as recited in dependent claim 
19 and the base claim 14, as amended. 

Applicants respectfully submit that, for the reasons described above, 
independent claim 14, as amended, patentably defines over the cited references and is, 
therefore, in condition for allowance, as are claims 15-23, depending therefrom. 



13 



CONCLUSION 

For the reasons detailed above, it is respectfully submitted all claims remaining in 
the application (Claims 1-23) are now in condition for allowance. The foregoing 
comments do not require unnecessary additional search or examination. No additional 
fees are believed to be required for this Amendment B, however, the undersigned 
attorney of record hereby authorizes the charging of any necessary fees, other than the 
issue fee, to Xerox Deposit Account No. 24-0037. 

In the event the Examiner considers personal contact advantageous to the 
disposition of this case, he/she is hereby authorized to call Mark Svat, at Telephone 
Number (216) 861-5582. 



Respectfully submitted, 




FAY, SHARPE, FAGAN, 
MINNICH & McKEErLLP 



Reg. No. 34,261 
1100 Superior Avenue, 7 th Floor 
Cleveland, Ohio 441 14-2579 
(216) 861-5582 
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